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Problem / Background / Context: 

Recruitment is a key challenge for researchers conducting any large school-based study. 
Random assignment studies pose greater challenges to recruitment because of the degree of 
control that researchers often require. Participants must be eager to implement the target 
intervention - and yet, must be equally willing to NOT implement the target intervention, 
depending on random assignment. Participants must be willing to give up control of an aspect of 
their school or classroom. Participation often involves obtaining coordinated commitments from 
multiple levels of a school system: teachers, principals, and district leaders. As a result, 
recruitment for randomized controlled trials requires a true partnership between researchers and 
the practitioners who design and implement the tested program. 

Further, recruitment often involves finding a cohort of new adopters of an intervention 
who will all agree to start at the same time - and few programs naturally have large enough 
cohorts joining en-mass with the desired timing. In order to have sufficient statistical power to 
detect at least medium-sized effects (effect sizes of .20 standard deviations or larger) and allow 
for attrition, researchers typically need to obtain samples of 30-60 teachers or schools. To 
preserve the integrity of the study, the research team needs to maintain teachers’ participation in 
their assigned treatment status and completion of data collection activities throughout the study. 
The seeds for success in these ongoing aspects of randomized controlled trials start during the 
recruitment phase. Attracting and retaining a coherent cohort is a key research challenge which 
begins with recruitment. 

This paper harvests lessons learned across 19 randomized controlled trials in education 
conducted by a leading research organization to identify strategies that have fostered successful 
recruitment efforts. 

Purpose / Objective / Research Question / Focus of Research/ Research Design: 

Our objective was to look across strategies used in a wide range of studies to build a 
framework for researchers to use in conceptualizing the recruitment process. Because of the 
number of RCTs that SRI International has conducted, the variation in the types of interventions 
studied, and the diversity of settings in which we have conducted our studies, we have a unique 
sample from which to build a framework that can inform the field. 

Population / Participants / Subjects/ Settings 

We report on our experiences in recruiting teachers for random assignment experiments 
in public primary and secondary schools. Our perspective is based on 19 current and completed 
randomized controlled trials in K-12 school settings. While this collection of studies is a 
naturally occurring sample, it is still a useful sample due to its diversity. The studies included in 
the sample address: 

• Varied topics, such as educational technology, literacy, mathematics, science, 

instructional materials, and teacher professional development, 

• Varied units of randomization, including the student-, teacher-, school-, and district-level, 

• Varied overall populations, which might focus on a particular state, specific schools or 

districts, or existing usage areas for the studied intervention across multiple states, 
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• Varied demographics including urban, suburban, and rural schools; higher- and lower- 
poverty schools; special populations of students (English Learners, students receiving 
special education services); students pre-K through 12th grade, 

• Varied kinds of participants, including both individual volunteers (i.e., where individuals 
could voluntarily enter the study and randomization occurred among those who 
volunteered) and those focused on entire organizations (e.g., principals decided their 
school should participate and teachers were expected to follow that decision), 

• Varied durations of experiments, from weeks to years. 

Improvement Initiative / Intervention / Program / Practice: 

Unlike the typical structured abstract, our initiative was to combine insights from many 
experimental projects, each with very different interventions, programs and practices. Unlike a 
typical meta-analysis, our purpose is not to bring the outcomes of these together, but rather the 
lessons learned about conducting a key phase of each project: recruitment. In order to 
accomplish this, we developed a six phase framework for collecting insights about the best 
practices in recruitment. The six phases are: study design, tailoring and packaging, planning the 
recruitment effort, design messages, running a recruitment campaign, and post recruitment 
follow up. 

Data Collection and Analysis: 

We collected data by making a list of 19 large, field-based randomized controlled 
designs, which SRI International has conducted. A set of leaders from these projects met first to 
draft an initial framework of the kinds of recruitment information we would like to extract across 
projects. For the most part, the desired knowledge was craft knowledge and was not available in 
written form. Consequently, we asked project leaders to identify the set of people who had the 
most experience with this craft knowledge, which included both some principal investigators as 
well as staff who have frequently been at the forefront of recruitment efforts. As knowledge was 
collected it was organized into a framework, which was necessarily revised to accommodate the 
insights that accumulated. A smaller group of leaders reviewed the resulting document and raised 
questions and issues, which were resolved by a second round of data collection. Finally, a 
revised document was prepared and reviewed by leaders across the projects, who agreed it 
represented a consensus on best practices in recruitment based on their experiences. 

Findings / Outcomes: 

Looking across the 19 studies, we identified six phases in the process for recruiting 
participants. We describe best practices identified for each phase briefly below; the presentation 
of this paper will be able to go into somewhat greater depth and provide examples and 
supporting evidence. 

Study Design. During the design stage, researchers need to balance factors that could 
improve the quality and value of their study with the size of the “ask” from participants, which is 
a central issue affecting how easy or difficult it will be to recruit participants for a study. These 
decisions center around the interplay between external validity (Is the program being used 
according to its model with a typical population?) and chances of maintaining internal validity 
(Can the team obtain sufficient implementation and participant retention?). As such, the 
decisions involve tradeoffs, and we recommend that recruitment decisions be made in 
collaboration with program developers and the leaders of the experimental design. 
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Key decisions include the unit of randomization (e.g., typical tradeoffs include the 
increased likelihood of contamination with teacher- as opposed to school-level randomization 
balanced by the need for a larger recruitment effort to obtain equivalent power with school-level 
as opposed to teacher-level assignment) and the use of convenience, vs. purposeful, vs. random 
sampling of populations (which will later be randomly assigned to specific treatments). A best 
practice is purposeful sampling that considers both the conditions likely to support strong 
implementation (for efficacy trials) and as well as desired generalizability of results to broader 
populations. On the program side, it is important to balance the duration, intensity, and impact of 
the program. While greater impacts may be more likely for longer and more intense programs, it 
goes without saying that it is far harder to recruit (and retain) teachers for an experiment that 
lasts longer and involves more substantial changes in work practice. Finally, researchers and 
program developers alike need to be judicious in collecting data to reduce costs and participant 
burden, while still garnering data necessary to understand implementation and impacts. 

Tailoring and Packaging the Intervention. To successfully recruit and engage 
participants for the duration of a study, researchers need to understand the potential intrinsic and 
extrinsic incentives — and disincentives — that will be in play during the experiment. For 
example, does the program increase or decrease teachers’ workloads in the short- and long-term? 
Even if it requires more work, is it aligned with broader individual or organizational goals that 
make the program inherently valuable? How can participants be enticed to participate in data 
collection? How great is the perceived harm of being assigned to the control condition likely to 
be? Incentives need to be balanced such that both treatment and control conditions remain 
motivated to remain in their treatment status and complete data collection throughout the study. 
Understanding the answers to these questions is the foundation upon which successful 
recruitment lies. A best practice often involves working with the program developers to refine 
the packaging of the intervention (e.g. the materials, training, etc.) to minimize the unnecessary 
complexity participants might experience, lower the threshold for teachers to engage, and to 
address issues that might lead to participant attrition. 

Planning a Recruitment Process. Recruitment is a process that unfolds in several stages 
over an extended period of time and requires the coordinated action of multiple staff members. 
As such, a good plan for the recruitment process is essential before actual recruiting begins. One 
best practice is to consider is the number of layers of the educational structure that need to be 
addressed through the recruitment process. Recruitment frequently requires building support at 
multiple levels (e.g., district, school, teacher, parent, student) and ways of working with each 
layer need to be tailored to concerns at that level. Because recruitment often takes a while, we 
recommend attention to the necessary lead-time, and best practices often involve allowing time 
for contingency planning when things do not go as well as expected at first. Another best 
practice is for the research organization to engage local organizations who already have credible 
and long-lasting relationships with potential participants as partners in the recruitment drive. 

Designing Messages. The framing of a research study that is effective in convincing peer 
reviewers to score a proposal highly has little to do with the best framing to attract recruits. 
Therefore, need to design customized recruitment messages before the recruitment process 
commences. A best practice is to develop a message that is clear and resonates with the 
perspectives of all the individual roles (e.g., principal, teacher, parent) whose support will be 
needed - framed around how they see their own needs, concerns, and the potential benefits of 
participating. Successful messaging is built on the understanding (described above) of incentives 
and disincentives in play. We have found that intrinsic motivations are greater around alleviating 
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pain (i.e., addressing real problems) as opposed to acquiring new niceties, and thus a best 
practice often involves focusing recruitment around a pain point being experienced by the target 
population (e.g. addressing new standards or a particularly difficult instructional challenge). 
Additionally, messages need to be low-jargon, balance portraying the picture of benefits with a 
realistic view of associated burdens. If recruitment materials downplay the requirements of 
participating in the study, later attrition is more likely. Finally, we have found it important to 
educate participants about random assignment in order to overcome negative (and, in our view, 
unfounded) concerns about the methodology writ large. 

Running a Recruitment Campaign. Recruitment can best be understood as a campaign. 
Like an election campaign, the goal is challenging, the process unfolds as unforeseen obstacles 
arise, and many people have to work together in a coordinated fashion to make it happen. At the 
center of the campaign, we have typically held weekly conference calls among all key 
stakeholders: the front-line recruiters, the intervention provider, and the researchers. This is 
necessary to handle emergent issues, to identify who needs to receive which communication, 
track potential participants through a ‘pipeline’ from initial contact through confirmed 
participation, and to keep the work appropriately coordinated. A best practice in the campaign is 
to maintain a database that tracks communications with each target participant (e.g. teacher, 
principal, district official) across the various recruiters who may interact with them and tracks 
their present status in the recruitment campaign (e.g. raw lead, expressed interest, planning to 
apply, applied, accepted). This database both allows focusing the recruitment effort but also 
keeping track of the emerging demographics of the sample, which can enable re-targeting further 
effort to obtain the a final sample with the desired characteristics. 

Post- Recruitment. Once recruitment ends, random assignment and baseline data 
collection typically need to begin quickly. The understandings, processes, and relationships the 
developed over the course of the recruitment process, however, have hopefully laid a strong 
foundation for maintaining participant engagement in the subsequent study. Careful planning 
must go in the transition from the recruitment process to the study on-boarding process. 

Conclusions: 

Recruiting for an RCT is particularly intense and challenging. Based on our experiences 
with recruiting for many projects, we have found it useful to conceptualize recruitment in phases, 
each of which has a detailed set of considerations appropriate to its essential work. Given that 
recruitment is the foundation of a successful RCT, careful attention to these processes from when 
the study is initially conceived until the last pieces of data are in, are critical to the conduct of the 
research. Consequently, we would advocate for more detailed attention to recruitment in research 
proposals; in the work of peer review panels and pre-award negotiations; and in the work of 
advisory committees. Based on our experience, a well -detailed recruitment plan should clearly fit 
the research design, explain how the intervention will be packaged to reduce the burden on 
participants, describe the overall timeline, staffing and project management structure for the 
campaign, give examples of how messages will be communicated to potential recruits, discuss 
strategies for handling contingencies that emerge during the campaign, and be thoughtful about 
how applicants will be welcomed into the study and retained as participants. In the presentation 
at SREE, we will discuss examples of these phases and invite the audience to share both their 
questions and their experiences. 
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